A primary design objective for Data-intensive User-
facing (DU) services for cloud and edge computing is to maximize
query throughput, while meeting query tail latency Service Level
Objectives (SLOs) for individual queries. Unfortunately, the
existing solutions fall short of achieving this design objective,
which we argue, is largely attributed to the fact that they
fail to take the query fanout explicitly into account. In this
paper, we propose TailGuard based on a Tail-latency-SLO-and-
Fanout-aware Earliest-Deadline-First Queuing policy (TF-EDFQ)
for task queuing at individual task servers the query tasks
are fanned out to. With the task queuing deadline for each
task being derived based on both query tail latency SLO and
query fanout, TailGuard takes an important first step towards
achieving the design objective. TailGuard is evaluated against
First-In-First-Out (FIFO) task queuing, task PRIority Queuing
(PRIQ) and Tail-latency-SLO-aware EDFQ (T-EDFQ) policies
by simulation. It is driven by three types of applications in
the Tailbench benchmark suite. The results demonstrate that
TailGuard can improve resource utilization by up to 80%, while
meeting the targeted tail latency SLOs, as compared with the
other three policies. TailGuard is also implemented and tested in
a highly heterogeneous Sensing-as-a-Service (SaS) testbed for a
data sensing service, with test results in line with the other ones.
more »
« less
Free, publicly-accessible full text available July 1, 2024
